To solve the problem that Circulant Structure of tracking-by-detection with Kernels (CSK) is difficult to adapt to the target scale change, a multi-scale kernel correlation filter classifier was proposed to realize the scale adaptive target tracking. Firstly, the multi-scale image was used to construct the sample set, the multi-scale kernel correlation filtering classifier was trained by the sample set, for target size estimation to achieve the goal of the optimal scale detection, and then the samples collected on the optimal target scale were used to update the classifier on-line to achieve the scale-adaptive target tracking. The comparative experiments and analysis illustrate that the proposed algorithm can adapt to the scale change of the target in the tracking process, the error of the eccentricity is reduced to 1/5 to 1/3 that of CSK algorithm, which can meet the needs of long time tracking in complex scenes.